Using Better Communication to Improve Programming by Demonstration

My research focuses on how a user communicates with a programming-by-demonstration (PBD) system. In particular, I experimenting with new nonverbal, direct manipulation techniques that will enhance a user’s expressiveness and subsequently will make it possible to infer a broader range of application behavior. The techniques include a new form of demonstrational interaction called nudges used to specify behavior. Complementing nudges is a special form of selection which is used to give the system hints by identifying significant objects. A new deck-of-playing-cards metaphor is also introduced for specifying useful effects such as randomness and sequencing. Other techniques use objects for annotating examples uch as behavior icons for manipulating and editing behaviors, and temporal ghosts to allow explicit references to past states. Finally, using guide objects is a technique for demonstrating constraints and hidden connections between objects. Complementing these new techniques will be an inferencing algorithm sufficiently powerful to convert data from all these sources into application behavior. Special attention will be given toward how hints are used to regulate inferencing. By fostering better communication between the user and the system, these techniques should allow the user to create highly interactive software with minimal programming expertise.